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ABSTRACT 


This study focuses on the retrospective evaluation of collaborative mLearning 
(CmL) Science module for teaching secondary school science which was designed 
based on social constructivist learning theories and Merrill’s First Principle of 
Instruction. This study is part of a developmental research in which computer- 
mediated communication (CMC) tools such asdiscussion forums, collaborative 
work spaces (wiki), and text messaging was employed. The user evaluation 
focuses on the retrospective evaluation of sixteen (16) Form 2 students as users 
on completion of the module. Data collection was from a surveys and interviews. 
However, to ensure reliability, the results were triangulated with the 
users’online communications on the forums, wikis, and text messages. The 
findings showedthat the CmLmodule has several strengths and weaknesses. 
However, the CmL module could be used for learning as it provided an 
opportunity for scientific inquiry in the search for information and through both 
online and face-to-face discussions while solving problems. In addition, the 
findings show that the CmLmodule could be used to encourage scientific inquiry 
for encouraging the nature of science. 
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INTRODUCTION 

The nature of science is the characteristic of scientific knowledge of the natural world which is derived through 
scientific inquiry (Lederman, 2012). Scientific inquiry through scientific processes is done in collaboration with other 
scientists (Abruscato, 2000; Hogan &Fisherkeller, 2005). Hence, science is a culture where the scientists are the 
practitioners of the culture, using the language of the culture to collaborate in scientific inquiry (Lederman, 2012). 

A similar discovery approach of scientific inquiry focusing on the nature of science, and not the content or 
scientific knowledge,should be used when teaching science. In order to do this, science learners need to be able to 
interact socially: to communicate and debate issues regarding science and society, as well as use science for their 
personal needs in life. Through their discussions and social interactions, the learners attempt to link the newly 
acquired knowledge with their existing knowledge, to interpret nature following certain rules and standards (DeWitt 
&Siraj, 2008; Galili, 2012). 

The language of science,scientific verbal knowledge, is important for building science concepts. The culture and 
social activity in scienceare important for all learners to build their knowledge and promote critical thinking (Galili, 
2012; Hoyle & Stone, 2000). While scientific terms can be defined formally, the language structures and the 
interpretation of the culture of rules in science are acquired informally (Galili, 2012). Social interactions in the culture 
of science enable the patterning and modeling of the language of science (Galili, 2012; Karpov& Haywood, 1998). 

In the Malaysian scenario, there is little time for social interaction in the science classroom. Teachers perceive 
that there is insufficient time to complete the science syllabus. The emphasis in most schools is on passing the 
standardized examinations. This has lead teachers to emphasize on the memorization of facts, rather than on the 
scientific processes (Chong, 2005). 


33 


www.mojet.net 










The Malaysian Online Journal of Educational Technology 


Volume f, Issue2 


Computer-mediated communication (CMC) tools can facilitate the communication and extend it outside the 
classroom (Anastopolou,Sharpies, Ainsworth, Crook, O'Malley, & Wright, 2011; Arrigo, Gentile, Taibi, Chiappone, 
&Tegolo, 2004; Capuano, Gaeta, Miranda, &Pappacena, 2004; Guzdial& Turns, 2000; Chang, 2010; Jeng, Wu, Huang, 
Tan, & Yang, 2010; Slotta& Linn, 2000; Saeed, Yang, &Sinnappan, 2009). CMC tools such as wikis, forums and text 
messaging enable informal discussions in science outside the formal classroom, and can be monitored by a facilitator 
or tutor. Research has shown that formal classroom learning can be supplemented with informal learning through 
technology use (Anastopolou et al., 2011). 

Collaborative learning occurs as the acquisition of knowledge, skills and attitude as a result of interactions in a 
group (Johnson & Johnson, 2004). When CMC tools are used, collaborative mlearning, which occurs as a result of 
interactions in a group, can happen at anytime and anywhere, even outside the formal classroom environment (Siraj& 
Alias, 2005; Siraj, 2005). Learning in this way can provide the social interaction for developing the culture of science 
(Galili, 2012; Karpov& Haywood, 1998). 

Although studies on the use of CMC tools for collaborative mlearninghave been conducted, most of the studies 
have focused on one or at most two CMC tools (Arrigo et al., 2004; Capuano et al., 2004; Guzdial& Turns, 2000; Saeed, 
Yang, &Sinnappan, 2009; Slotta& Linn, 2000). This study seeks to investigate the effectiveness of a combination of 
three CMC tools on learning with the collaborative mlearning (CmL) Science module. In this study, the three CMC tools 
of the wiki, discussion forum and text messaging would be combined to develop the CmL Science module. 

PURPOSE OF THE STUDY 

The purpose of this research is to evaluate theCmL Science module developed based on Merrill's principles of 
instruction for learning. This study seeks to answer the following research questions: 


1. What are the strengths and weakness of the CmL Science module? 


2. How do learners learn with the CmLScience module?. 


THE LANGUAGE OF SCIENCE 

Language enables the culture and thinking processes for the construction of knowledge (Galili, 2012; Hoyle & 
Stone, 2000). Scientific verbal knowledge, or the knowledge for communicating in the science vocabulary, is required 
for understanding and communicating in science so that dialogues to build the culture of science can occur (Hoyle & 
Stone, 2000; Karpov& Haywood, 1998). Learners can then construct meaningful phrases using scientific terms to 
communicate their thoughts and develop science concepts through inquiry (Hoyle & Stone, 2000; Karpov& Haywood, 
1998). Student-centered discussion develops critical thinking skills, as arguments are resolved to reach a mutual 
understanding (Hoyle & Stone, 2000; Karpov& Haywood, 1998). 

Scientific knowledge is a culture, and cultural content knowledge which refers to the nature of science needs to 
be taught through discourse (Galili, 2012). The vocabulary and structures of the language of science is a cultural aspect 
acquired through discourse during social interactions. Vygotsky's view is that scientific knowledge and procedures 
should not be taught directly but should be constructed by learners during discussions to buildunderstanding of 
science concepts (Karpov& Haywood, 1998). 

Social interactions and dialogue internalizes learning as learning is situated (Chang, 2010; Jeng, Wu, Huang, 
Tan, & Yang 2010). Computers and mobile phones are the cultural tools in the external environment whilethe 
language, the environment, and the CMC tools are the abstract social tools which assist in developing. The cultural 
and social tools encourage the process of cognitive change in the learner (Gredler, 1997). 

In the CmL module, knowledge creation is affordedthrough scientific inquiry and arguments to generate new 
ideas (Laru, Jarvela, &Clariana, 2012; Rogers, Connelly, Hazlewood, & Tedesco 2010; So, Tan, &Tay 2012). At the same 
time, learners are scaffold by their instructor and peers (Boticki, Looi, & Wong, 2011; Timmis, 2012). Hence, 
learners'social interaction in the culture of science promote inquiry through peer collaborationto develop their 
understandings (Vygotsky, 1981). 
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DESIGN OF INSTRUCTION FOR LEARNING SCIENCE 


The CmLScience module designed for teaching science is based on Vygotsky's social constructivist learning 
theories.Hence, activities for learners to build their personal understanding through discussions, opportunities for 
patterning and modeling with individualized support,as well as scaffolding to address the difficulties learners 
experience in trying to comprehend the language of science are provided. Social interactions are encouraged to 
provide opportunities to link the science knowledge withlearners' prior experience in a culture of building meaningful 
science knowledge (Brown, 2006). 

The use of CMC tools, such as wikis and discussion forums, for teaching science is not new. The Knowledge 
Integration Environment (KIE), a platform for storage of web resources and scaffolding through tips and guidance for 
the activities, employed anonline asynchronous discussion forum for collaborative mLearning in science (Slotta& Linn, 
2000). Another platform for collaborative mLearning, CaMILE, has discussion forums and encourages communication 
and collaboration on science projects (Guzdial& Turns, 2000). Both these platforms wereeffective in learning science. 

Textual learning objects have been pushed to learners through short messaging system (SMS) for language 
instruction (Capuano et al., 2004). Text messages have been used combined with a wiki (Arrigo et al., 2004) and with 
discussion forums (Rau, Gao, & Wu, 2008). A combination of CMC tools for learning science improved examination 
performance and motivated learners can be used(Rau, Gao, & Wu, 2008). 

In the Malaysian context, there is a lack of research in the use of a combination of CMC tools on a CmL platform 
for teaching science. Hence, it is believed that the CmL Science module in this study, which will employ three CMC 
tools,will have similar results to other studies. The CmL module can provide experiences for building scientific 
knowledge and concepts to encourage the scientific inquiry process. Further, it is hoped that this study will provide 
insights in the use of a CmL environment for science instruction. 

FIRST PRINCIPLES OF INSTRUCTION 


The design of this module was based on the First Principles of Instruction (Merrill, 2002).These design- 
orientated principles could be used to solve real-world problems for learning environments in any delivery system 
(Figure 1). Instruction should then take into account the four phases of learning: (a) activation of prior experience; (b) 
demonstration of skills; (c) application of skills; and (d) integration of these skills into real world activities. Application 
of the First Principles of Instruction in the CmL module is summarized in Table 1. 
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Figure 1: First Principles of Instruction 
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Table 1: Application of First Principles of Instruction 


First Principles of Instruction (Merrill, 2002) 


Application in the collaborative mLearning 
module 


Learning is promoted when learners are engaged 
in solving real-world problems 

Learning is promoted when existing knowledge is 
activated as a foundation for new knowledge 

Learning is promoted when new knowledge is 
demonstrated to the learner 

Learning is promoted when new knowledge is 
applied by the learner 

Learning is promoted when new knowledge is 
integrated into the learner's world 


An online problem task which is meaningful to the 
learner would be solved as group-work on the 
wiki. 

Questions on the online discussion forum to 
activate prior knowledge. 

An instructional module on the webpage with links 
to other web pages, videos, and interactive 
software is provided. 

Questions on the online discussion forum to test 
application of knowledge. 

Quiz pushed through text messages to the learners 
and group-work on the wiki used to asses learners' 
integration of knowledge. 


METHODOLOGY 

DESIGN OF THE STUDY 

The study is part of a developmental research which focuses on the final user-evaluation of theCmL module 
(MuhamadSabri, Nor Aziah, Zawawi&NuruIhuda 2012; Richey 1997;Wang &Hanafin 2005). The evaluation is based on 
the usability evaluation method framework where retrospective evaluation was employed. (Chai & Chen, 
2004;Norlidah Alias, SaedahSiraj&VanithaThanabalan, 2011). Retrospective evaluation focuses the users' reactions on 
completion of the module (Gertler,Martinez,Premand, Rawlings&Vermeersch, 2011). 

An urban secondary school in the state of Selangor, which had an enrolment comprising of the multi-racial 
communities of Malaysia, was selected. The CmL Science module was implemented withsixteen Form 2 students. On 
completion of the tasks in the module, the users' reaction towards the usability of the module for learningwas 
determined. A survey of the usability of the module, and an interview was conducted. The responses in the 
retrospective evaluation were triangulated with data from online communications in the module for reliability. 

DEVELOPMENT OF THECMLSCIENCE MODULE 

The CmLScience module on the topic of Nutrition was designed based on social constructivist learning 
theoriesand the First Principles of Instruction (Merrill, 2002).Learnershad many misconceptions in the topic of 
Nutrition. Secondary school children were confused about the concept of food: water and vitamins are not food but 
were considered as food (Lee &Diong, 1999). In addition, students rated Nutrition as the most difficult topic with a lot 
of factual knowledge in science (DeWitt &Siraj, 2007). 

The CmLmodule was hosted on a website, with links to content, videos, animations, and CMC tools.The 
activities were designed to incorporate a main problem task which had to be solved collaboratively on a wiki. In 
addition, other smaller problem tasks to activate, demonstrate, apply and integrate knowledge were discussedon a 
forum, while individualized quiz were pushed through text messaging. 

The web page and its links to content and interactive tools provided the demonstration of knowledge while 
application was through the questions on the discussion forum. For integration of knowledge, learners had the 
opportunity to reflect and use the knowledge learnt through questions on the discussion forum and SMS Quiz. 
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Figure 2: Tools in the collaborative mLearning module 
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Module Implementation 

The CmLScience module was implemented with a group of sixteenForm 2students from the selected school. 
The participants were from a pool of volunteers and equal numbers of high, medium and low-achievers in science 
were selected.Most of the students in the selected school (81.6%) owned a mobile phone, and had access to 
computers (63.9%). However, laptops with a mobile broadband modem for internet access,and mobile phones for text 
messaging after school hours in school were provided to enable students who required access. 

Data Collection and Analysis 

The CmL module was implemented with the group of students. In retrospective evaluation, data collection is 
done after the completion of the module. The data collected from the survey of the usability of the CmL module was 
analysed using descriptive statistics. The participants were interviewed on the usability of the CmL module. The 
transcript of the interview was coded and analysed for the emergent themes. 

The data was analyzed to find the strengths and weaknesses of the CmL module and how the students in the 
context of the study were learningin the CmL environment. Triangulation of data was done through the analysis of the 
online communications in the module for reliability. 

RESULTS AND DISCUSSION 

The research questions on the strengths and weakness of the CmL module and how learners learn with the 
CmL Science module is discussed below.The findings of this study may provide insights into the effects of CmL and 
whether it could be used for teaching science. 

STRENGTHS AND WEAKNESS OF THE MODULE 

The qualitative analysis of the data showed that the CmLmodule had several strengths as it improved 
understanding, encouraged the search for information, and learning through online and face-to-face discussions, as 
well as interaction with content with immediate feedback (Table 2). In addition, the users were motivated and 
interested in learning. 
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Table 2: The strengths of learning withthe CmLScience Module 


No. Categories 

Evidence 

Understanding science 

Because when 1 do this module, it improves my knowledge. In 
addition, the module helped in my revision of the topic (Mat) 

1 think there’s nothing that 1 disliked. Cause if 1 do this 
module, it’s good for my knowledge. In addition, the module 
helped in the revision of the topic (Shah) 

Because we like to talk to our friends, chit chat, and then 
‘masukkepalajuga’ 

Encouraging search for 
information 

It makes me open my book. 1 won’t open it if 1 don’t have 
exams. (Nabil) 

1 read my friends answers (Siti) 

And 1 can do a lot of research online (Nadirah) 

1 like it because it makes me open my science book more often 
(Shah) 

Learning through online 
discussions 

Well, one thing for sure, with the questions, you can always 
ask people, or you can refer to your Science text book. At 
least it helps us to brainstorm a bit. (Shah) 

1 remember better after doing the discussions. There’s this one 
question on the walls of the organ in the objective section. 1 
picked C as the answer. And in the subjective questions, they 
asked about the name of the movement for food in the organs, 
so 1 recalled it- peristalsis (David) 

Sometimes I’m online at MSN, then 1 ask my group members 
what 1 should do. Then they tell me (Nadirah). 

Learning through face to 
face discussions 

We do discussions in a group, we don’t use the computer first. 

We talk and sit face-to-face, and one person will write what 
we discussed about it (Nailah) 

Learning through 

interaction with content 

Honestly, 1 think the SMS Quiz makes me remember (Shah) 

1 did not post any answer. 1 don’t know the answer. But 1 did 
see the others’ answers. It helped a bit. (David) 

Like it because can give our opinions and comments (Nadirah) 

Interest in learning 

It’s interesting. 1 have something to keep me occupied. (Nabil) 

Well once in a while when 1 don’t have anything else better to 
do, or if 1 have questions asking, about the online question, 
then 1 can at least have the (SMS) questions to keep me 
company (Shah). 

You get to learn differently, and not just from the book. So, 
you get to use the internet, and then ya, something different. 

(Alia). 

1 expected fun, 1 enjoyed internet (Jeanne) 

Immediatefeedback 

Our phone is just inside our (pockets) so we can just reply 
immediately... 


There were also weaknesses observed in the implementation of the CmL module (Table 3). Some of the users' 
faced technical difficulties and were not too sure on the implementation. In addition, the use of English as the 
medium of instruction was raised. The users' perception of knowledge, and other external factors such as school work 
and parental restrictions on the use of internet interfered with the learning and the process of scientific inquiry. 
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Table 3: The weakness of learning withthe CmL Science Module 


No.Categories 


Technical difficulties 


Language - English 
Perception of learning 


Other Priorities 

Parental control 
Group work 


_ Evidence _ 

I don't know how to use. (Alvin) 

I dislike doing it because I don't have a Yahoo account (Chan) 

I am a little confused with Yahoo Tech groups.(Nadirah) 

(Freewebs- Participation in this forum was easier and can post 
discussions) 

The online stuff kinda hard as I can only access the internet 
sometimes. Depends on the connection. (Nabil) 

Yes, some language problems. Grammar - I don't understand 
(David) 

I don't read others' answers. Because if I read their answers, 
maybe they'll think I'm copying them (Nadirah). 

I don't think if I see the others answers I will learn (David) 

I was preoccupied with homework (Ann) 

Because sometimes I'm not in the mood to (Gwen) 

My mother don't let me use so much internet (Shahirah) 

Dislike it because I need group members (Shah) 


LEARNING SCIENCE 

From the survey, most of the users (84.3%) believed their understanding in science improved after completing 
the module, while the rest were undecided (17.7%). The use of the CmL moduleencourages learning as it improves 
understanding (Table 2). While searching for informationin either online or print materials, demonstration of 
knowledge was exhibited and this provided experiences for modeling science inquiry.Learners had the opportunity to 
observe and pattern the scientific verbal and cultural content knowledge in the nature of science (Galili, 2012) through 
the problem-solving task. 

In addition, the social interactions through discussions, both online and face-to-face, using the language of 
science contributed to the culture of scientific inquiry in learning. During the discussions, patterns of language use and 
processes of inquiry are formed while learners collaborate on their tasks, while quietly observing the discussions 
among their peers. Learning was a social activity where elements in the environment, including answers of the 
participants could be artifacts to "mediate" learning within the culture of scientific knowledge building. 

Lack of participation could be due to technical difficulties in using the discussion forum, and difficulty with the 
medium of instruction: English.However, it was verified that learnerswho did not participate did read their peers' 
postings. Learners perceived that reading others' postingswas considered copying. Their beliefwas that learning was 
individualized and a not process to be shared with others. 

In general, the CmL module motivated the learners as they were interested to use it. Learning takes place 
formally and informally when the social interactions through other learners' answers and mediated learning. 

IMPLICATIONS AND CONCLUSIONS 

The findings of this study indicate that the CmLScience module enables learning science through the process of 
scientific inquiry(Lederman, 2012). Learners participated in the collaborative discussions in science using CMC tools, 
indicating that CMC tools can be used to support a scientific learning culture (Anastopolou et al., 2011; Arrigo et al., 
2004; Guzdial& Turns, 2000). The communications which were part of the process of achieving a shared goal in the 
task (Johnson & Johnson, 2004) enabled the thinking processes is actually beneficial for learning for constructing 
science knowledge (Hoyle & Stone, 2000). Communication and learning can be conducted virtually outside the science 
classroom. 

The use of a combination of CMC tools have been shown to be effective for learning (Arrigo et al., 2004; Rau et 
al., 2008). In the CmL module, a combination of three CMC tools was used for teaching science. Providing a 
combination of tools allowed learners to respond and use the tool most convenient or most preferred for learning. 
The learners' preference for different tools might be related to different learning styles of the learner and could be a 
possible area for further research.In addition, only sixteen users participated in this study. The findings of the study 


39 


www.mojet.net 







MOJET 


The Malaysian Online Journal of Educational Technology 


Volume f, Issue2 


are still relevant as the participants were selected from different science abilities. However, future studies could be 
done to determine the effectiveness of the CmL module with a larger sample. 

When CMC tools are used, collaborative mlearning is encouraged (Guzdial& Turns, 2000; Slotta& Linn, 2000) as 
learners to work in groups to solve tasks. The findings of this study reinforce the fact that collaborating in scienceas 
acultural and social activity is important for learners to develop their understandings and promote critical thinking 
through the language of science (Galili, 2012; Hoyle & Stone, 2000). The patterns and modeling of the language of 
science used during the discussions could be developed(Galili, 2012; Karpov& Haywood, 1998). 

The 'silent observers' lurking in the background did not seem to participate, observeing the social interactions, 
were also involved in the informal learning process when they viewed others' answers. Their observations would 
contribute to the patterns of science culture for modeling (Karpov& Haywood, 98). Further investigation could be 
done to determine if the lack of participation was because of the inaccurate perception of the nature of science 
whereby the learners expected only one correct answer. These learners have to be given scaffolding and 
encouragement to participate in the interactions for developing the culture of scientific inquiry in the nature of 
science(Galili, 2012; Lederman, 2012). Considerations may have to be made for the social and cultural tools to include 
more exemplars and guidance for discussion questions (Gredler, 1997). 

A discovery approach in teaching science which emphasizes the nature of science and scientific inquiry should 
allow social interactions. In this study, the social interactions and collaboration, both formally and informally, have 
contributed to building scientific verbal knowledge (Abruscato, 2000; Hogan &Fishkeller, 2005). Learning occurred 
during the process of communication and scientific inquiry was encouraged through the problem- solving tasks. 
However, thedevelopment of the true nature of science in which knowledge is subjective requires a more rigorous 
design. 


The findings are important as in the transformation of the education system there is a need to design effective 
methods of building knowledge in science. In the CmLmodule a discovery approach to allow the processes of learning 
science through communication and discussion is encouraged. Social interaction in which the learner debates issues 
and integrates with their existing knowledge is useful in building meaningful knowledge. The use of suitable 
instructional design principles, which allow for problem-solvingin learning can be employed(Merrill, 2002). 

In summary, the CmLScience module is can be used forlearning science to address the learning needs in 
science. In addition, learning through social interaction may be extended to the teaching of other subjects as well. 
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